Maximizing entropy over Markov processes

Maximizing Entropy over Markov Processes

The channel capacity of a deterministic system with confidential data is an upper bound on the amount of bits of data an attacker can learn from the system. We encode all possible attacks to a system using a probabilistic specification, an Interval Markov Chain. Then the channel capacity computation reduces to finding a model of a specification with highest entropy. Entropy maximization for pro...

ADK Entropy and ADK Entropy Rate in Irreducible- Aperiodic Markov Chain and Gaussian Processes

In this paper, the two parameter ADK entropy, as a generalized of Re'nyi entropy, is considered and some properties of it, are investigated. We will see that the ADK entropy for continuous random variables is invariant under a location and is not invariant under a scale transformation of the random variable. Furthermore, the joint ADK entropy, conditional ADK entropy, and chain rule of this ent...

The Entropy of Recursive Markov Processes

Projections Maximizing Tsallis Entropy Projections Maximizing Tsallis Entropy

We consider a small closed physical systems. It has xed energy and if the energy is a power of its generale coordinates then the distribution of any single coordinate will follow a distribution that maximizes Tsallis entropy. For many aspects of entropy can be discussed for such small systems with out going to the thermodynamical limit. By letting the number of degrees of freedom tend to in nit...

On Hidden Markov Processes with Infinite Excess Entropy

We investigate stationary hidden Markov processes for which mutual information between the past and the future is infinite. It is assumed that the number of observable states is finite and the number of hidden states is countably infinite. Under this assumption, we show that the block mutual information of a hidden Markov process is upper bounded by a power law determined by the tail index of t...